Adaptive step-length selection in gradient boosting for Gaussian location and scale models

Abstract Tuning of model-based boosting algorithms relies mainly on the number iterations, while step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive for location, scale and shape (GAMLSS), imbalanced updates predictors, where some distribution parameters are updated more frequently than others, can be problem that prevents submodels to appropriately fitted within limited iterations. We propose an approach using adaptive (ASL) determination non-cyclical algorithm Gaussian location models, important special case wider class GAMLSS, prevent imbalance. Moreover, we discuss properties ASL derive semi-analytical form avoids manual selection search interval numerical optimization find optimal step-length, consequently improves computational efficiency. show competitive behavior proposed approaches compared penalized maximum likelihood in two simulations applications, particular cases large variance and/or variables observations. In addition, underlying concept also applicable whole GAMLSS framework other one predictor like zero-inflated count brings up insights into choice reasonable defaults simpler (Gaussian) models.